Over the past years, there has been an exponential increase in the demand for video services. Video data dominates Internet video traffic and is predicted to increase much faster than other media types in the years to come. Cisco predicts that video data will account for 79% of Internet traffic by 2018 and mobile video will represent two-thirds of all mobile data traffic by 2018. Well accustomed to a variety of multimedia devices, consumers want a flexible digital lifestyle in which high-quality multimedia content follows them wherever they go and on whatever device they use. This imposes significant challenges for managing video traffic efficiently to ensure an acceptable quality-of-experience (QoE) for the end user, as the perceptual quality of video content strongly depends on the properties of the display device and the viewing conditions. Network throughput based video adaptation, without considering a user's QoE, could result in poor video QoE or wastage of bandwidth. Consequently, QoE management under cost constraints is the key to satisfying consumers and video monetization services.
Digital videos are subject to a wide variety of distortions during acquisition, processing, compression, storage, transmission, reproduction, and display, any of which may result in degradation of visual quality. For applications in which videos are ultimately to be viewed by human beings, the only “correct” method of quantifying visual image quality is through subjective evaluation. In practice, however, subjective evaluation is usually too inconvenient, time-consuming and expensive. Objective video quality assessment (VQA) methods may automatically predict the quality assessment behaviours of humans viewing the video signals.
VQA methods have broad applications 1) in the evaluations and comparisons of the quality of videos and the performance of different video acquisition, processing, compression, storage, transmission, reproduction, and display methods and systems; 2) in the control, maintenance, streaming, and resource allocation of visual communication systems; and 3) in the design and optimization of video acquisition, processing, compression, storage, transmission, reproduction, and display methods and systems.
The simplest and most widely used prior art VQA measure is the mean squared error (MSE), computed by averaging the squared intensity differences of distorted and reference image pixels, along with the related quantity of peak signal-to-noise ratio (PSNR). The MSE and PSNR are simple to calculate and are mathematically convenient in the context of optimization. But they are not very well matched to perceived visual quality [1]. The most famous and representative state-of-the-art prior-art methods include the structural similarity index (SSIM) [2,3], the multi-scale structural similarity index (MS-SSIM) [4], the video quality metric (VQM) [5], and the motion-based video integrity evaluation index (MOVIE) [6]. All of them have achieved better quality prediction performance than MSE/PSNR. Among them, the best tradeoff of quality prediction performance and computational cost is obtained by SSIM and MS-SSIM [7]. Despite this, none of them considers the differences between the viewing devices of the end-users, which are an important factor of the visual quality-of-experience of the end users. For example, the human quality assessment of the same video can be significantly different when it is displayed on different viewing devices, such as HDTV, digital TV, projectors, desktop PCs, laptop PCs, tablets, and smartphones, and many more. Prior-art techniques ignore such differences and do not contain adaptive frameworks and mechanisms that can adjust themselves to the changes of viewing device parameters. Moreover, the quality analysis information provided by prior art methods is limited. For example, VQM and MOVIE do not supply spatially and temporally localized quality maps, SSIM does not produce quality maps at different scales, and SSIM and MS-SSIM do not take into account temporal distortions.
Therefore, what is needed are improvements to the methods and systems for objective perceptual video quality assessment which overcome at least some of the limitations of the prior art.